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(57) Abstract 



A remote data collection system, which may be used in a vehicle such as an aircraft or a ground vehicle, includes a directional 
sensor, such as one or more cameras, for sensing a characteristic of interest and providing sensor data. The system further includes a global 
positioning system (GPS) receiver for providing GPS data representative of the position of the sensor, an inertial measurement unit (IMU) 
for providing IMU data representative of the attitude of the sensor, a processing unit and a storage unit. The processing unit determines 
geographic data referenced to the sensor data in response to the GPS data and the IMU data. The processing unit may utilize an error model 
to determine IMU errors which may be used in determining the geographic data with high accuracy. The sensor data and the geographic 
data are stored in the data storage unit for subsequent use. The system may include a stabilized platform on which the sensor and the 
IMU are mounted. The stabilized platform is rotated about at least one axis of rotation to control the sensing direction of the sensor as ihe 
vehicle moves. 
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AIRBORNE IMAGING SYSTEM 

Field of the Invention 

This invention relates to vehicle-mounted sensing systems and. more particularly, to high 
5 resolution, low cost sensing systems which operate on a moving vehicle. The invention is 
particularly useful for airborne imaging, but is not limited to this use. 

Background of the Invention 

The remote sensing market originated with early satellites and goals of global monitoring of 
10 terrestrial activities. However, the cost of data and the response times for obtaining data has 
limited the broad applicability of imagery for use in day-to-day business operations. 
Nevertheless, for relatively small scale users, such as fanners, city planners, utilities managers 
and forest managers, the computerized geographic information system is recognized as the 
information integration tool of the future. Geographic information systems include computer 
15 tools for locating geographic coordinates of points within images, for overlaying maps and 

images, and for making quantitative measurements, such as areas, distances and precise locations 
of objects, from the images. Applications as diverse as (1 ) a farmer selecting chemical 
application strategies based upon expected crop yield predictions, (2) a tax assessor directing a 
manual inspection of an observed dwelling addition, or (3) assessing timber yields and timber 
20 harvest costs from multiple property tracts are but a few of the numerous potential application^ 
of imagery and geographic information systems. 

The profit structure of the agricultural industry is heavily dominated by the use ot chemicals 
(fertilizers, pesticides and herbicides), with a trend toward the use of more chemicals. I iowc\ er. 
since the EPA and world environmental organizations recognize long term hazards of unabated 
25 chemical treatments, chemical regulations are throttling the agricultural business. Of e^uai 
significance as populations increase, more land is cleared for farming, and reduced food > id J- 
per acre lead to higher deforestation and even greater threats to the environment. The rc^^ni/ci 
answer to these global scale problems is metered usage of chemicals, such that the cherr..cal^ are 
used where they are of maximum benefit. Metered chemical distribution systems are ;n a:uc 
30 use. However, the data to define the spatial metering values are lacking. Multispec;ral ::r.at-er. 
with sub-meter resolution and spatial registration is required. The imagery must be a*. aii-Ke r 
low cost and with short response times to requests for imagery. 
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A satellite system known as the SPOT satellite, sponsored by the government o, France. ,s 
r eprcse„,a„veofour TC „ l opera„o„. l s, Kl H.ec a pab i U. i es. Tins system provides »«« 
resohnion panchromatic tmagery or 20 meter reso,u,ion imagery in ,he visible/near mfrared 
bands Geodetic regis,ra,ion is accurate ,„ 1 5 meters in the U.S. where ground control pomts are 
ptentM and well surveyed. Experience has shown that the response ttme for nnagery quests ,s 
usually no better than >0 days. The drawbacks to sateUite imagery inc.ude the potcnna, for cioud 
coverage <*> of the earth is cioud covered,, high costs and the tnflexibility of sateUite tmagery 
coUection systems. Custom tailored resolutions, spectral bands and responsiveness cannot be 

provided bv satellite systems. 

Airborne imagery is commonly used by many small scale users. Benefits of airborne 
imagery collection tnclude tailorable reso,ution. response time and data processing methodology. 
Drawbacks include higher costs and the need to contract f„, a dedicated aircraft, nigh, crew and 
post-mission processing system. 

A variety of airborne sensing and survey systems have been disclosed in the pnor an. A 
, survey system for obtaining geophysica, data win, aircraft using real time differential operate 
of the globa, positioning system is diseased in U.S. Pa, No. 4, 8 ,4.7„ tssued March 21, ,9»9 
Olsen et al. An airborne system using rwo color video cameras and an IR imager head mounted 
be,ow the fuselage of an aircraft is disclosed in U.S. Pat No. 5,166.7*9 issued November .4 
,992 to Myrtck. Latitude and longitude information obtained from the global postuomng system 
,„ is recorded on each image frame. A system including a CCD camera and a globa, poshiontng 
S ystem receiver fo, recordtng an tmage signal and position information on magne.tc „pe ,s 
disclosed in U.S. Pat. No. 5.267.042 issued November 30. ,993 to Tsuchiya et a,. A techntooe 
for airborne imagtng wherein multiple overlapping images are superimposed by observing a 
stationarv object tha, appea* in adjacent images is diseased in U.S. Pat. No. 5. 247.56 tssued 
,5 September. 1993 to Ciampa. A technique for generating high resolution images from a CCD 
camera in an aircraft is disclosed in U.S. Pal. No. 5.251.037 issued October 5. 1993 to 
Busenberg. A technique for generattng high resolution vidicon aerial tmages is dtsclosed ,n U.S. 
Pa, No 5 -70.756 .ssued December .4. 1993,0 Busenberg. An airborne con.our mapptng 
svstem is *sc,osed in U.S. Pat. No. 3,9,,70 tssued June 22. ,965 ,0 Lusti, e, a, A ,e=hn,que 

„ for remote sensing using inenia, navigation systems and the globa, position.ng system for 
j0 lorrcn * p^ngramrnetnc 

georeferencing of remotely sensed data ,s desenbed by k.P. Schwarz 
EagmS£I ^^ 59.No. ll.Novembe, 1993.pp. 1667-1674. 
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The examples above are characterized by components integrated tightly to the aircraft so 
that a dedicated aircraft is required. None of the prior art airborne imaging systems have been 
practical from the viewpoint of small scale users with respect to cost, resolution, flexibility and 
response time. 

5 

Summary of the Invention 

According to the present invention, methods and apparatus for remote data collection are 
provided. The invention is used in a vehicle such as an aircraft or a ground vehicle. A remote 
data collection system in accordance with the invention comprises a directional sensor for 

10 sensing a characteristic of interest and providing sensor data representative of the characteristic 
of interest, a global positioning system (GPS) receiver for providing GPS data representative of 
position and velocity of the sensor, an inertial measurement unit (IMU) for providing IMU data 
representative of attitude rate and acceleration of the sensor, a processing unit and a data storage 
unit. The processing unit determines IMU errors in response to the IMU data and an IMU error 

1 5 model. The processing unit also determines geographic data referenced to the sensor data in 

response to the GPS data, the IMU data and the IMU errors. The sensor data and the geographic 
data are stored in the data storage unit for subsequent use. The sensor may comprise one or more 
cameras, and the sensor data may represent images. 

According to another aspect of the invention, a remote data collection system for use in a 

20 vehicle comprises a stabilized platform, a directional sensor rigidly mounted to the stabilized 
platform for sensing a characteristic of interest and providing sensor data representative of the 
characteristic of interest, an inertial measurement unit (IMU) rigidly mounted to the stabilized 
platform for providing IMU data representative of attitude of the sensor, a control unit responsive 
to the IMU data for rotating the stabilized platform about at least one axis of rotation with respect 

25 to the vehicle to control the sensing direction of the sensor as the vehicle moves, a global 
positioning system (GPS) receiver for providing GPS data representative of position of the 
sensor, a processing system responsive to the GPS data and the IMU data for determining 
geographic data referenced to the sensor data, and a data storage unit for storing the sensor data 
and the geographic data for subsequent use. 

30 The control unit may comprise a gimbaL a support member rigidly mounted to the vehicle, a 

first motor connected between the support member and the gimbal for rotating the gimbal about a 
first axis relative to the support member and a second motor connected between the gimbal and 
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the stabilized platform for rotating the stabilized platform about a second axis relative to the 
gl mba, The direction of the sensor is stabilized with respect to the first and second axes as the 
vehicle moves. In a preferred embodiment, the vehicle comprises an aircraft, and the control una 
stabilizes the soring direction of the sensor in a vertical orientation with respect to pitch and roll 
s of the aircraft. 

AH facets of the invention are optimized for low operational cost and rapid response. The 
sensor and the IMU are integrated to a readily detachable vehicle component (e.g. cargo door, so 
that no modifications to the vehicle are required. The IMU used for sensor stabilization ,s also 
used for attitude determination and navigation, thus reducing cost. A mission plannmg/vehtck 
steering command system is included to allow one man operation for cost reduction and 
operational ease of use. 



10 



PrK ^r?""P tiftn of f he drawings 

For a better understanding of the present invention, reference is made to the accompan> ,ng 
, 5 drawings, which are incorporated herein by reference and in which: 

FIG. 1 is a block diagram of an embodiment of an airborne imaging system in accordance 

with the present invention; 

FIG. 2 is a pictorial diagram showing the components of the airborne imaging system o, 

FIG 1 * 

20 FIG. 3 is a pictorial diagram illustrating the airborne imaging system of FIG. 1 installed on 

an aircraft; 

FIG. 4 is a top schematic view showing the stabilized platform assembly mounted on uk 
door of an aircraft; 

FIG. 5 is a top view showing the stabilized platform assembly in more detail: 
, 5 FIG 6 is an exploded view of the stabilized platform assembly ; 

FIG. 7 is a software flow diagram that illustrates operation of the airborne imaging ^ -n. 

and 

FIG. 8 is a pictorial diagram that illustrates an example of an airborne survey miS s,on 

f ailed Descriptio n 

A block diagram of an embodiment of an imaging system in accordance with .he ,u — ■» 



is shown 



in FIG. 1 . The imaging system includes an airborne data collection system ■ • 



1 ■< a!l J 
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ground processing workstation 12. and makes use of a GPS around station 14. The airborne data 
collection system 10 is typically mounted in an aircraft and is used for obtaining images of a 
prescribed survey area of the earth. For example, the imaging system may be used to obtain 
images of a prescribed agricultural area or forest area. As described in detail below, the ground 

5 processing workstation 12 is used to define a trajectory to be followed by the aircraft in order to 
obtain images of the survey area with complete coverage. The ground processing workstation 12 
may also be used for post-mission processing of image data and for administrative functions. 

The airborne data collection system 10, in general, includes a directional sensor for 
generating sensor data, a global positioning system (GPS) receiver for providing GPS data 

10 representative of the position of the sensor, an inertial measurement unit (1MU) for providing 
1MU data representative of the attitude of the sensor, a processing unit responsive to the GPS 
data and the IMU data for providing geographic data referenced to the sensor data, and a data 
storage unit for storing the sensor data and the geographic data. The geographic data establishes 
the ground coordinates of the sensor data with high resolution. The airborne data collection 

15 system 10 may include a stabilized platform assembly for stabilizing the direction of the sensor 
during aircraft flight. The data collection system of the invention may be used in ground 
vehicles as well as in aircraft. 

Referring again to FIG. K an embodiment of the airborne data collection system 10 includes 
cameras 20. 22 and 24. The cameras 20, 22 and 24 may, for example, be charge coupled device 

20 (CCD) cameras with filters having different spectral responses. The cameras 20, 22 and 24 
supply image data and synchronization signals to a frame grabber 28. The frame grabber 28 
supplies image data representative of individual images obtained by each of the cameras to a 
system computer 30. A disk storage unit 32 connected to the system computer 30 is used for 
storage of image data and geographic data. The airborne data collection system 10 further 

25 includes an inertial measurement unit 40 that provides IMU data to the system computer 30 
through an IMU interface 42. The inertial measurement unit 40 is rigidly mechanically 
connected to cameras 20, 22 and 24 and typically senses acceleration and rotation rate with 
respect to three coordinate axes. A GPS receiver 46 receives positioning signals from GPS 
satellites through a GPS antenna 48. The GPS receiver 46 also receives positioning signals trom 

30 the GPS ground station 14 through a data link antenna 50 and an RJF modem 52. The GPS 

receiver 46 supplies GPS data and a GPS clock to the system computer 30. As known in the an. 
the GPS data accurately represents the position of GPS receiver 46 and therefore represents the 
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position of cameras 20. 22 and 24. The system uses different GPS for steering the vehicle to 
accuracies of 1-3 m. A kinematic GPS processing procedure is applied post-m.ssion to allow 
determination of position accuracy to the 10 cm level. 

When the airborne data collection system 10 includes a stabilized platform assembly, at leas, 
one stabilizing motor is provided. In the example of FIG. 1. a roll motor 60 stabilizes the 
cameras ^0 22 and 24 with respect to aircraft roll, and a pitch motor 62 stabilizes cameras 20. 22 
and 24 with respect to aircraft pitch. Roll motor 60 is energized by system computer 30 through 
a motor amplifier 64. and pitch motor 62 is energized by system computer 30 through a motor 
amplifier 66. Each of the motors 60 and 62 includes an encoder which provides to system 
, computer 30 a signal representative of motor angle with respect to a reference angle. Operation 
of the stabilized platform assembly is described in detail below. A power supply 70 receives 
aircraft power, typically 28 volts, and supplies operating power to the components of the 
airborne data collection system 10. A graphical display 72 is connected to system computer 30. 
As described below, the display 72 provides commands to the pilot when the aircraft deviates 
5 from a preplanned trajectory over the survey area. 

FIG 2 is a pictorial diagram that illustrates a preferred configuration of the airborne data 
collection system 10. A stabilized platfonn assembly includes cameras 20, 22 and 24, IMU 40, 
motors 60 and 62 and additional components described below. In a preferred embodiment, the 
stabilized platform assembly 80 is mounted to a cargo door 82 of an aircraft (not shown m FIG. 
, 0 The airborne data collection system 10 further includes an electronics urut 84. The 

electronics unit 84 includes system computer 30, disk storage unit 32, GPS receiver 46, RF 
modem 5^ frame grabber 28. IMU interface 42, motor amplifiers 64 and 66, power supply 70 
and displav 72. The electronics unit 84 is interconnected to stabilized platform assembly 80 by a 



cable 86 



FIG 3 is a pictorial diagram illustrating installation of the airborne data collects system of 
the present invention in an aircraft 90. As Seated above, stabilized platform assembly 80 ,s 
preferablv mounted on cargo door 82. Electronics unit 84 is posiuoned in the cargo area ot the 
aircraft 90 GPS antenna 48 may be mounted on the upper surface of the aircraft, and data l,nk 
antenna 50 may be mounted on the unders.de of the aircraft. The pilot display 76 is posmoncd 

30 for convenient viewing by the pilot. 

FIG 4 is a pictorial diagram of the stabilized platform 80 mounted on cargo door 82. !G. 
shows the stabilized platform assembly in more detail, and FIG. 6 shows an exploded v.ew ot th 
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stabilized platform assembly. Cameras 20, 22 and 24 are rigidly mounted to a stabilized 
platform 100. The stabilized platform 100 extends through an opening in cargo door 82 such that 
cameras 20. 22 and 24 are located externally of the aircraft. The opening in cargo door 82 
provides sufficient clearance to permit movement of stabilized platform 100 with respect to the 

5 aircraft. In a preferred embodiment, the opening is a vertically-oriented slot to permit pitch and 
roll stabilization of the cameras. The cameras 20. 22 and 24 are protected by a cowling 102 that 
is open at the bottom. The IMU 40 is rigidly mounted to a portion of stabilized platform 100 
within the aircraft. A support member 102 is mounted to an inside surface of cargo door 82, and 
roll motor 60 is secured to an inwardly-extending portion of support member 102. The shaft of 

10 roll motor 60 is connected to a gimbal 106. Pitch motor 62 is mounted to gimbal 106. and the 
shaft of pitch motor 62 is secured to stabilized platform 100. As shown in FIGS. 5 and 6. 
cameras 20 f 22 and 24 are mounted in a camera housing 120, which is attached to stabilized 
platform 100. Stabilized platform 100 is connected to gimbal 106 by a pivot pin 122 and is 
connected to the shaft of pitch motor 62 by a flange 124. Gimbal 106 is connected to the shaft of 

15 roll motor 60 by a flange 126. 

When the roll motor 60 is energized, cameras 20, 22 and 24, stabilized platform 100, IMU 
40, gimbal 106 and pitch motor 62 are rotated with respect to a roll axis 1 1 0. When the pitch 
motor 62 is energized, cameras 20, 22 and 24, stabilized platform 100 and IMU 40 are rotated 
with respect to a pitch axis 112. The nominal direction of flight of the aircraft is indicated in 

20 FIG. 4 by arrow 114. 

In an example of the stabilized platform assembly 80, the roil motor 60 was a Hathaway 
type HT03802 brushless DC motor, and the pitch motor 62 was a Hathaway type HT02301 
brushless DC motor. The motor amplifiers 64 and 66 were Hathaway type BLC048 motor 
amplifiers. 

25 In operation^ the IMU 40 senses changes in velocity and angle in three coordinate directions. 

Since cameras 20, 22 and 24 and IMU 40 are rigidly connected to stabilized platform 100. 
velocity changes and angle changes sensed by IMU 40 represent velocity and angle changes of 
cameras 20. 22 and 24. IMU data representative of the velocity and angle changes is supplied to 
system computer 30. The system computer 30 uses the angle changes to determine deviations of 

30 the attitude of the cameras from a desired attitude. These deviations are used to generate error 
signals which are supplied through motor amplifiers 64 and 66 to roil motor 60 and pitch motor 
62. respectively. The roll motor 60 rotates cameras 20. 22 and 24 with respect to roll axis 1 10. 
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and pitch motor 62 rotates cameras 20. 22 and 24 with respect to pitch axis 1 12. so as to maintain 
a desired attitude. Typically, the cameras 20. 22 and 24 are maintained in a vertical attitude with 
respect to the earth's surface. However, other boresight directions may be utilized. Furthermore, 
the cameras may be scanned, for example, with respect to the roll axis 11 0 to obtain images of a 
s wider strip on each pass over the survey area. 

The stabilized platform assembly 80 provides advantageous operation of the imaging 
system. Because the cameras 20, 22 and 24 are stabilized, typically in a vertical orientation, the 
spacing between adjacent aircraft passes over the survey area can be increased without risking 
loss of coverage between images in adjacent passes. This is possible because it is not necessary 
,0 to account for inadvertent aircraft roll in determining the spacing between passes. By increasing 
the spacing between adjacent passes of the aircraft trajectory, the time and cost for completing a 
given survey is reduced. In an alternative approach, the cameras are scanned with respect to the 
roll axis at a rate relative to the aircraft speed which permits imaging of a wider strip than » 



possible with stationary cameras. 

the stabilized platform assembly 80, shown in FIGS. 4-6 and described above, provides 
stabilization with respect to the pitch and roll axes of the aircraft. In another configuration, the 
stabilized platform assembly is simplified to provide stabilization with respect to the roll ax.s 
only In this configuration, the shaft of roll motor 60 may be connected directly to stabilized 
platform 100 so as to rotate cameras 20, 22 and 24 with respect to roll axis 1 10. It is be.ieved 
, that stabilization of the cameras 20. 22 and 24 with respect to the yaw axis (perpendicular to axes 
U 0 and 1 12) would not provide substantial benefits in the operation of the imaging system. 

Mounting of the stabilized platform assembly 80 on the aircraft door provides significant 
practical advantages in operation of the imaging system. In general, it is desired to install the 
imaging system of the present invention in arbitrary aircraft. One alternative is to install the 
, cameras in a hole cut in the floor of the aircraft. However, this requires a special modification to 
the aircraft and requires certification of the installation by the FAA. Such a hole ,s unlikely to be 
acceptable to manv aircraft owners. Wing mounting of the camera assembly is undesirable for 
similar reasons. Mounting of the stabilized platform assembly on the cargo door provides an 
attractive solution. Cessna 1 72 aircraft, for example, have a cargo door that ,s easily removable. 
,0 Other small, four-passenger commercial aircraft have a similar cargo door which may be 

modified for installation of the stabilized platform assembly. A 6x6 inch hole is cut in the lower 
interior portion of the dual wall aluminum door structure. The support member 102. having a 
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using a carrier frequency of 460 MHz. This system provides approximately a 100 mile rad.us of 
coverage with a 1 5 watt transmitter and omnidirectional datalink antenna 50. 

The system computer 30 may comprise an industry standard model PCI single board 
computer, which utilizes a P5 1 50 MHz processor. I/O functions are handled by a model ATC40 
, carrier board available from Greenspring, which provides four Industry Pack (IP) board slots for 
tailoring the I/O functions performed by the board. One IP board is the IP-ADIO available from 
Greenspring, which provides analog-to-digital, digital-to-analog and discrete digital I/O 
functions. This IP board receives IMU data and the GPS clock and supplies motor control 
signals to the motor amplifiers 64 and 66. An IFservo board decodes the motor encoder signals 
10 received from roll motor 60 and pitch motor 62. 

The disk storage unit 32 must have sufficient storage volume and a sufficient data transfer 
rate to store image data supplied by the frame grabber 28. Consider an airborne mission 
requiring one foot per pixel image resolution and an aircraft speed of 100 knots at one frame per 
second. The 640 x 480 pixel single camera image frame provides a 65% overlap between images 
, 5 (480 pixel dimension along direction of motion). At one byte per pixel for each of the three 
cameras, a data storage rate of 0.92 megabyte per image, or 0.92 megabyte per second, is 
required. A conservative 1.5 megabyte per second storage rate is used as the nominal image 
storage transfer rate specification. 

A nominal time for an imaging survey mission may be 3 hours, with over 2 hours assumed 
20 for actual image collection. The remaining time is spent flying to and from the survey area and 
for turnarounds after completion of each swath. Two hours of imaging will generate a 7.2 
gigabyte imagery file at a 1 .0 megabyte per second storage rate. The GPS data and IMU data 
recorded during the mission (0.04 megabyte per second for 3 hours) contributes only 0.4 
gigabyte of additional storage, for a total of 7.6 gigabytes. One example of a suitable disk unit .s 
^5 the Seagate Elite-9. having 9 gigabytes of storage and 1 1 milliseconds access time. The 

standard SCSI disk drive interface allows storage throughput up to 1 .5 megabytes per second.. 

The imaging system of the present invention utilizes direct digital photography and digital 
storaae of spatially registered imagery. Other airborne video systems use a videotape system as 
the airborne image storage medium. This allows several hours or imagery to be captured at a 30 
30 Hz image rate. However, videotape does not capture the full resolution or the full dynanuc range 
of the CCD camera systems. All known airborne video systems offer videotape of the surveyed 
terrain, with frames tagged with GPS positions. 
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box structure, is used to cam' torque from the pitch and roll motors. A torque transfer stiffener 
1 30 (FIG. 6), 14 inches in length, is part of the support member 102 and transfers the roll motor 
torque in the vertical plane of the door. A vertically-oriented slot is cut in the door to allow the 
camera support portion of the stabilized platform 1 00 to pass through the door to the exterior of 

5 the aircraft. When the imaging system of the invention is to be used in an aircraft, the standard 
cargo door is removed and is replaced with a cargo door having a preinstalled stabilized platform 
assembly. The electronics unit 84 is placed in the cargo area of the aircraft. 

The cameras used in the imaging system may include three compact monochrome CCD 
cameras. Such cameras are available from numerous suppliers. A preferred camera is the Sony 

0 XC-7500, which provides 640 x 480 pixel resolution in non-interlace (progressive scan) mode. 
The cameras typically use a 16 mm lens with an f-stop of 2.8. Different filters can be utilized in 
the camera lens to provide different spectral responses. For example, red, green, blue and near 
infrared filters may be utilized to obtain different information regarding the survey area. A color 
image can be formed by using red, green and blue filters. The frame grabber may be a Mu-tech 

5 model M-1000 which allows access to up to 4 cameras simultaneously. 

The imaging system has been described thus far with reference to a configuration utilizing 
three cameras. It will be understood that any number of cameras may be utilized. More 
generally, any sensor having a boresight direction for sensing may be utilized for data collection. 
Thus, for example, the sensor may be a laser system, an atmospheric pollution sensor, a thermal 

0 camera, a radar system or any other suitable sensor. 

The IMU may be a Honeywell H-1700 system, which has an accuracy characterized by 10° 
per hour gyro accuracy. While higher accuracy IMLTs are available, the cost is also higher. In 
order to utilize a low-cost IMU with moderate accuracy, an error model of the IMU is utilized as 
described below. 

5 The GPS receiver 46 may comprise an eight-channel Motorola Encore airborne unit, and 

the GPS ground station 14 may comprise an 8-channel Motorola Encore differential GPS base 
station. The Motorola Encore is a C/A code unit with capability for using differential corrections 
transmitted by the base station. The GPS receiver 46 is connected by a coaxial cable lo GPS 
antenna 48. installed on the upper surface of the aircraft. A true kinematic GPS system is a 

0 preferred implementation to achieve accuracies of 10 cm or better. 

The RF modem 52. which provides the differential GPS datalink to GPS ground station 14. 
may be a Pacific Crest RFM 96S radio modem, capable of two-way communication at 9600 baud 
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Thc system uses a display to provide steering cues to the pilot to maintain the appropriate 
flight line. An Accuphoto system provided by Genysis Comm, Inc. is used for this purpose. The 
Accuphoto system provides a software tool for use in planning the mission, resulting in a 
software file defining the mission profile in GPS coordinates. The onboard GPS receiver is then 
5 used to provide pilot cues via a simple LCD display indicating need for a left/right correction and 
the magnitude of the correction. 

The configuration described above wherein stabilized platform assembly 80 is mounted on a 
cargo door of the aircraft provides a number of advantages in operation of the imagine system. 
However, other configurations may be utilized within the scope of the present invention For 
10 example, the stabilized platform assembly may be omitted from the imaging system. In this 
configuration, the cameras are rigidly mounted to the aircraft, and the IMU data is used lo 
compensate the image data for aircraft pitch, roll and yaw. Furthermore, the cameras or other 
sensors are not necessarily mounted on the aircraft door. For example, the cameras may be 
mounted in a hole in the floor of the aircraft, in a pod beneath the aircraft or on one of the wings. 
15 The ground processing workstation 12 performs survey mission planning and post-mission 

processing. The ground processing workstation 12 may be implemented using a PC-based 
graphical workstation and commercially available geographic information system tools (GIS » 
such as Arcview available from ESRL Several mission planning functions are provided by the 
ground processing workstation. It allows the user to view a digital line graph (map> database. 
20 available from commercial sources, depicting the survey area of interest. The user selects the 
boundary points to define the survey area selected for mission coverage. An aircraft traiec;»'~> is 
computed from a takeoff point to the survey area with sufficient passes over the surve> area u • 
provide a high probability of coverage of the selected area at a specified resolution and aire ran 
flighttime. Multiple missions are prescribed where required. The aircraft trajectory :s displaced 
25 to the user superimposed over a digital map of the area of interest. The aircraft traieeu>r> . 
defined by waypoints, and image collection start/stop times are stored on a floppy di*k for 
entering into the airborne data collection system 10. System parameters such as camera setup 
parameters (frame resolution and angular field of view), aircraft parameters (endurar.ee. >. eu--eir. 
and turn radius) and mission descriptors (airport location and percent frame overlap i ::m\ re 
30 modified. An example of a mission trajectory 250 is illustrated in FIG. 8. 

Post-mission processing functions of the ground processing workstation 12 inci.uit* 
extracting data from the disk storage unit 32 of the airborne data collection system 1 • • 
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registration of the individual images onto a geodetic reference frame and combining the images 
into a contiguous imagery file stored in a standard GIS format. The post-processing functions 
use GIS tools that are similar to those used for mission planning. The mass storage media in the 
ground processing workstation 12 is compatible with the disk storage unit 32 in the airborne data 
collection system 10. Files in the disk storage unit 32 may be copied to the mass storage system 
in the ground processing workstation 12. The ground processing workstation accesses the stored 
files and registers each file individually into the ground plane. Multiple neighboring individual 
images from a single mission can be overlaid onto a common geodetic grid. All or a selected 
subset of the image frames may be mosaiced onfo a geodetic reference grid. GIS tools may be 
used to scroll, zoom and perform measurements on the mosaiced imagery. Imagery operations 
may be performed individually or on weighted combinations of the images from the three 



cameras. 



In one operating mode, the end user makes arrangements to lease an aircraft and performs 
mission planning. Using the ground processing workstation 12, the user lays out the survey area 
5 to be imaged by selecting boundary points on a digital map. As indicated above, the ground 
processing workstation considers the endurance and turning properties of the aircraft to be used, 
the base airport location and camera parameters. This allows the automatic design of a three- 
dimensional trajectory for the aircraft, with image collection points selected for ideal coverage of 
the survey area. The trajectory, or multiple mission trajectories, is presented to the user for 
20 approval. Higher resolution requirements require more passes and possibly additional missions. 
A three-hour mission with two hours of the flight time collecting image data provides over 
20.000 acres (32 square miles) of image coverage at a one foot image resolution. 

Following the trajectory design stage, the user obtains from the ground processing 
workstation a floppy disk that contains the digital specifications of the trajectory (X, Y. Z 
25 position versus time) and the image collection points. The airborne data collection svstem is 
installed on the aircraft, and a checkout of all subsystems is performed automatically . Upon 
valid checkout, the aircraft is ready to begin the mission. The pilot display leads the pilot 
through the mission from takeoff to landing, although the pilot can exit and re-enter the 
trajectory waypoint files at any time, if desired. The display also provides the current status ot 
30 the mission, for example, flight legs completed, time to next turn, loss of GPS lock, or the . 
occurrence of any anomalies which might result in loss of data. 

During image frame recording over the survey area, image data from the cameras 20. 22 and 



WO 97/35166 



PCT/TJS97/04668 



-13- 

24 is stored on the disk storage unit 32. The stabilized platform assembly maintains the cameras 
in a vertical orientation as described above. GPS data, representative of position of the cameras, 
and IMU data, representative of attitude of the cameras, is simultaneously stored on the disk 
storage unit 32. Each image frame has corresponding GPS data and IMU data, so that the image 

5 data may be spatially registered with high accuracy. 

After the mission is completed, the airborne data collection system may be deinstalled from 
the aircraft. The electronics unit 84 can be connected to the ground processing workstation 12, 
so that the stored data may be transferred to the ground processing workstation. The individual 
image frames are transferred to the storage media in the ground processing workstation and are 

10 registered in the ground plane using the GPS data and IMU data stored with the images. The 
image data can immediately be registered because position coordinates for each image pixel are 
known. This allows preparation of a contiguous registered image of the survey area and review 
of this image on the workstation using standard GIS tools. Maps showing roads, cultural 
features, hydrology and the like can easily be overlaid on the image. The user can now use the 

1 5 image in any desired manner. 

In the above scenario, the end user was responsible for all aspects of the survey mission. In 
other scenarios, an airborne imaging service may be established to support a higher volume 
operation. In this case, the service organization may offer mission planning support to users or 
may accept mission description disks from users who operate their own ground processing 

20 workstations. For example, an individual farmer or chemical supplier may request once per 

week imagery of his acreage to precisely time harvest and/or chemical application for maximum 
yield. 

The software in the system computer 30 of the airborne data collection system 10 is required 
to perform the following functions. The IMU data is processed to provide a strapdown 

25 navigation solution propagating the position, velocity and attitude of the camera axes. The 
strapdown navigation solution is combined with the GPS velocity data to provide a transfer 
alignment resulting in the attitude of the camera boresight relative to North. East and down. The 
three CCD cameras are commanded to obtain imagery in a synchronous manner with the GPS 
data and the IMU data. Precise GPS timing is used to synchronize all data collection functions. 

30 A trajectory manager monitors the current aircraft position relative to the desired trajectory and 
provides commands to the pilot indicating the degree of error in horizontal and vertical planes. 
The two-axis stabilization system uses the measured camera boresight attitude. IMU rotation 
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rates and motor encoder values to control the camera axes to point in a commanded direction, 
nominally down. The image frame data. GPS data. IMU data, attitude solution and system he.p 
status are logged on the disk storage system. 

A block diagram that illustrates the interrelationship of the software modules in the airborne 
data collection svstem is shown in FIG. 7. A strapdown navigation routine 200 propagates the 
position velocitv and attitude of the IMU coordinate axes forward in time at a data rate ot 100 
Hz usine digital measurements of change in velocity and change in angle. The inputs to the 
routine 200 are the IMU data samples at a 100 Hz rate and initialization values for the IMU 
attitude Additionally, attitude error values are input from the transfer alignment Kalman filter 
, -mp The outputs are ( 1 ) latitude, longitude and altitude, (2) North. East, down components of 
velocity, and (3) attitude Euler angles (roll, pitch and yaw) relating the IMU axes to the North. 
East and down axes. 

The transfer alignment Kalman filter 202 merges the GPS velocity measurement and the 
strapdown navigation routine output to produce an estimate of the error in the IMU axes attitude 
s computation. Inputs include the GPS velocity measurements and the strapdown navigation 
solution svnehronized in time. Additionally, the lever arm (in aircraft body axes) from the GPS 
antenna phase center to the IMU location is required. Finally, an IMU error mode. 204 
representing the statistics of the IMU errors is also required. The transfer alignment process 
utilizes a Kalman filter formulation based upon the IMU error model. The outputs include the 
,0 IMU attitude errors which are supplied to the strapdown navigation routine 200 as correcuon, 
Errors in the IMU gyro and accelerometer instruments are logged to address the IMU in-fhght 
performance. 

A sensor borestght stabilization module 208 processes IMU attitude rates, motor encoder 
values and IMU attitude data to control the pitch and roll motors so as to properly point the 
, 5 camera boresight to the desired direction. Inputs include motor encoder values. IMU rotation 
rates strapdown navigation attitude. Euler angles and timing stgnals from the IMU interrupt. 
The stabilization module includes two linear control systems identical in structure but having 
different gains to accommodate the different inertias presented to the motors. Stabilization ,s 

u «..ith the IMU interrupt. A conventional 

performed at a 100 Hz data rate synchronized v.ith the imu inu.« y> 

. • t ■ „„,. H The nroDOrtional term comes from the pitch or 

30 proportional integral derivative control i* used. I he proportion 

, iVnm the IMU rate gvro measurements, 

roll attitude errors, and the derivative terms come Irom the imu r<u fc . 

r.» i Knth the Euler aneles and the rotation rates to 
Coordinate transformations must be applied to both the tuier an fc i 
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account for the specific Euler angle set mechanized by the gimbal. The control gains are selected 
by a knowledge of the various control inertias, motor gain, and the amplifier gain and the 
selected bandwidth of the control loops. The pitch and roll bandwidths are selected at 10 Hz. 
The motor encoder data is not normally used within the control loop. However, this data is used 

5 to determine the orientation of the IMU axes relative to the aircraft for lever arm calculations and 
to determine the proximity of the gimbals to their mechanical limits. 

An image projection module 210 manages camera image frame collection and buffer 
storage. Inputs include frame time synchronization from the IMU 100 Hz interrupt, image 
memory addresses, kinematic GPS position and attitude angles from the strapdown navigation 

10 routine 200. The image synchronization is controlled by the IMU interrupts at 100 Hz with 
nominal frame rates of 1 to 2 Hz, i.e. 50 to 100 IMU samples between frame collection events. 
The frame collection commands use Mu-tech routines which provide frame triggering and 
synchronization of the three cameras. The outputs are memory mapped image frames for the 
three CCD cameras. 

15 The real-time software consists of three modules: 1) the strapdown navigation routine 200. 

2) the transfer alignment routine implemented in Kalman filter 202, and 3) the gimbal command 
routine implemented in the sensor boresight stabilization module 208. 

The strapdown navigation routine 200 consists of integration of 6-degree-of-freedom 
equations with a body-fixed coordinate system. The body coordinate system has the z-axis fixed 

20 to the camera boresight axis, the x-axis nominally pointed forward and the y-axis to the right of 
the motion. The internal coordinate system used for navigation is the Earth-Centered- Earth - 
Fixed (ECEF) system. Accelerations and rotation rates are integrated from the initial 
assumptions of attitude and using the GPS measured velocity components. The equations 
consider a WGS-84 datum for all computations for compliance with GPS. The strapdown 

25 computations are performed at a 100 Hz rate, which coincides with the availability of the IMU 
data. 

The transfer alignment routine implements a 22 state Kalman filter with stages of 
Covariance Propagation and state/covariance update at each measurement. The filter states 
include velocity errors, attitude errors, acceleromeler biases, gyro biases, accelerometer scale 
30 factor errors, gyro scale factor errors with each of these error terms containing x. y. and /. 
components (18 individual terms). An additional state is used to represent the time latency 
between the GPS and IMU measurement devices. Three additional states are used to represent 
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the time-integral of the velocity (average velocity, over a 200 msec window pnor to each OPS 
one pulse-per-second < 1PPS) time point. This velocity average is used to model exactly the 
functioning of the specific GPS recetver used in the preferred embodiment. The Kalman niter 
propagates elements of the covariance matrix and state between each 1PPS GPS time and 
5 performs a covariance and state update at each GPS time. The resulting attitude errors and . MU 
instrument errors are fed back into the strapdown navigation routine 200 to act as a continuous 
source of calibration. This allows use of small, low-cost 1MU devices which are currentK heme 

manufactured by several vendors. 

The pitch and roll motors are controlled by a conventional proportional, integral. uenvanve 
,0 (PID) controller implemented in the gimbal command routine. The rate gyros from the IMU 
provide the necessary rate feedback and the transfer alignment, coupled with the strapdown 
navigation routine, provides the position feedback. The three PID gains are derived from 
knowledge of the inertias and desired bandwidth of the closed loop system. This pointing s> stem 
differs from conventional systems in that the IMU used for navigation is placed on the inner 
,5 gimbal of the stabilized platform, which is also directly attached to the camera/sensor package 
This is enabled by availability of small, low-cost IMU components, direct drive servo motors and 
camera sensors. 

While there have been shown and described what are at present considered the preterm, 
embodiments of the present invention, it will be obvious to those skilled in the art that vanou, 
,0 changes and modifications may be made therem without departing from the scope ot the 
invention as defined by the appended claims. 
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What is claimed is: 

5 1 . A remote data collection system for use in a vehicle as the vehicle moves, said system 
comprising: 

a sensor for sensing a characteristic of interest and providing sensor data representative of 
the characteristic of interest, said sensor having a sensing direction: 

a global positioning system (GPS) receiver for providing GPS data representative of 
10 position of said sensor under moving conditions: 

an inertial measurement unit (IMU) for providing IMU data representative of attitude of said 
sensor; 

a processing unit responsive to said IMU data and an IMU error model for determining 
IMU errors, and responsive to said GPS data, said IMU data and said IMU errors for determining 
1 5 geographic data referenced to said sensor data; and 

a data storage unit for storing said sensor data and said geographic data for subsequent use. 

2. A sensing system as defined in claim 1 wherein said sensor comprises a camera and said 
sensor data represents an image. 

20 

3. A sensing system as defined in claim 1 wherein said sensor comprises a charge coupled 
device (CCD) camera and wherein said sensor data represents an image. 

4. A sensing system as defined in claim 1 wherein said sensor comprises a plurality of CCD 
25 cameras, each having a different spectral characteristic, and wherein said sensor data represents 

images in different spectral ranges. 

5. A sensing system as defined in claim 1 wherein said vehicle is an aircraft and wherein said 
sensing system is used for airborne imaging of a predetermined region. 

30 

f>. A sensing system as defined in claim 1 wherein said GPS receiver comprises a differential 
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GPS receiver. 

7. A sensing system as defined in claim 1 wherein said GPS receiver comprises a kinematic- 
capable GPS receiver. 

8. A sensing system as defined in claim 1 further comprising a stabilized platform to which 
said sensor and said IMU are rigidly mounted and a control unit responsive to said IMU data for 
rotating said stabilized platform about at least on* axis of rotation with respect to the vehicle to 
control the sensing direction of said sensor as the vehicle moves. 

9. A sensing system as defined in claim 8 wherein said vehicle comprises an aircraft and 
wherein said control unit includes means for rotating said stabilized platform about pitch and roll 
axes with respect to the aircraft to maintain the sensing direction of said sensor substantially 
vertical during flight. 

10. A sensing system as defined in claim 9 wherein said sensor, said IMU and said stabilized 
platform are affixed to a door of the aircraft. 

11 a sensing system as defined in claim 10 wherein said sensor comprises a plurality of CCD 
20 cameras, each having a different spectral characteristic. 

1 2. A sensing system as defined in claim 9 further comprising means responsive to a desired 
aircraft trajectory and said GPS data for indicating deviations of the aircraft from the desired 
aircraft trajectory. 

13. A remote data collection system for use in a vehicle as the vehicle moves, said system 

comprising: 

a stabilized platform; 

a sensor rieidlv mounted to said stabilized platform for sensing a characteristic of interest 
30 and providing sensor data representative of the characteristic of interest, said sensor having a 
sensine direction: 



15 
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an inertial measurement unit (IMU) rigidly mounted to said stabilized platform ibr 
providing IMU data representative of attitude of said sensor; 

a control unit responsive to said IMU data for rotating said stabilized platform about at least 
one axis of rotation with respect to the vehicle to control the sensing direction of said sensor as 
5 the vehicle moves: 

a global positioning system (GPS) receiver for providing GPS data representative of 
position of said sensor; 

a processing system responsive to said GPS data and said IMU data for determining 
geographic data referenced to said sensor data; and 
0 a data storage unit for storing said sensor data and said geographic data for subsequent use. 

14. A sensing system as defined in claim 13 wherein said control unit comprises a gimbal, a 
support member rigidly mounted to the vehicle, a First motor connected between said support 
member and said gimbal for rotating said gimbal about a first axis relative to said support 
5 member and a second motor connected between said gimbal and said stabilized platform for 
rotating said stabilized platform about a second axis relative to said gimbal, wherein the sensing 
direction of said sensor is stabilized with respect to said first and second axes as the vehicle 
moves. 

0 1 5. A sensing system as defined in claim 14 wherein said vehicle comprises an aircraft and 
wherein said control unit stabilizes the sensing direction of said sensor in a vertical orientation 
with respect to pitch and roll of the aircraft. 

1 6. A sensing system as defined in claim 1 3 wherein said control unit comprises a support 
5 member rigidly mounted to the vehicle and a motor connected between said support member and 
said stabilized platform for rotating said stabilized platform about an axis of rotation relative to 
said fixed frame, wherein the direction of said sensor is stabilized with respect to said axis of 
rotation as the vehicle moves. 

o 1 7. A sensing system as defined in claim 16 wherein said vehicle comprises an aircraft and 
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whcrein said control unit stabilizes the sensing direction of said sensor in a vertical orientation 
with respect to roll of the aircraft. 

18. An imaging system for use in an aircraft, said imaging system comprising: 
5 a stabilized platform; 

a camera system rigidly mounted to said stabilized platform for providing image data, said 

camera system having a boresight direction; 

an inertial measurement unit (IMU) rigidly mounted to said stabilized platform for 
providing IMU data representative of attitude of said camera system; 
1 o a control unit responsive to said IMU data for rotating said stabilized platform about at least 

one axis of rotation with respect to the aircraft to control the boresight direction of said camera 
system during flight; 

a global positioning system (GPS) receiver for providing GPS data representative of 
position of said camera system; 
, 5 a processing unit responsive to said GPS data and said IMU data for determining geographic 

• data referenced to said image data; and 

a data storage unit for storing said image data and said geographic data for subsequent use. 

19. An imaging system as defined in claim 18 wherein said camera system comprises a plurality 
20 of CCD cameras, each having a different spectral response. 

20. An imaging system as defined in claim 1 8 wherein said aircraft 

includes a cargo door and wherein said camera system, said IMU and said stabilized platform are 
mounted to said cargo door. 

* 25 

21 . An imaging system as defined in claim 18 further comprising means responsive to a desired 
aircraft trajectory and said GPS data for indicating deviations of the aircraft from the desired 
aircraft trajectory. 

30 22. An imaging system as defined in claim 1 8 wherein said GPS receiver comprises a 
differential GPS receiver. 
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23. An imaging system as defined in claim 18 wherein said GPS receiver comprises a 
kinematic GPS receiver. 

24. An imaging system as defined in claim 18 wherein said control unit includes means tor 

5 rotating said stabilized platform about pitch and roll axes with respect to the aircraft to maintain 
the boresight direction of said camera system substantially vertical during flight. 

25. An imaging system as defined in claim 1 8 wherein said processing unit further cDmpnsc^ 
means responsive to said IMU data and an 1MU error model for determining 1MU errors and 

10 wherein^aid processing unit is responsive to said GPS data, said IMU data and said I N 1 1 : error> 
for determining said geographic data with high accuracy. 
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